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AFOSR  Final  Technical  Report 
Intermediate  Levels  of  Visual  Processing 

Ken  Nakayama 
Grant  #  F49620-92-J-0016 
Award  period;  10/1/1991-10/31/1994 

Our  major  research  accomplishments  over  the  past  grant  period  are  summarized 
below: 

1.  We  completed  a  comprehensive  theoretical  account  of  visual  surface 
representation  based  on  a  new  understanding  of  many  of  our  recently  reported 
perceptual  phenomena.  In  particular,  we  determined  the  degree  to  which  the 
"generic  view  principle"  can  explain  these  new  findings  and  also  extended  its 
domain  to  a  large  number  of  perceptual  phenomena  (Shimojo  and  Nakayama, 

1992) .  We  apply  the  generic  sampling  principle  which  indicates  that  the  visual 
system  acts  as  if  it  were  viewing  surface  layouts  from  generic,  not  accidental 
vantage  points.  Through  the  observer's  experience  of  optical  sampling,  which 
can  be  characterized  geometrically,  the  visual  system  makes  associative 
connections  between  images  and  surfaces,  passively  internalizing  the  conditional 
probabilities  of  image  sampling  from  surfaces.  In  turn,  this  enables  the  visual 
system  to  determine  which  surface  a  given  image  most  strongly  predicts.  Thus, 
visual  surface  perception  can  be  considered  as  inverse  ecological  optics  based  on 
learning  through  ecological  optics.  Because  of  its  neglect  of  prior  probabilities 
and  higher  level  knowledge,  visual  surface  perception  deviates  from  strict 
Bayesian  notions  of  inference. 

2.  We  demonstrated  that  tasks  which  were  thought  to  be  mediated  by  the 
outputs  of  receptive  field  mechanisms  were  actually  mediated  at  a  more 
intermediate  level,  at  surface  representation.  Heretofore,  visual  search,  visual 
texture  segregation,  and  motion  perception  were  thought  to  be  determined  by 
image  properties.  Experimental  studies  of  visual  search  indicated  that  the 
conspicuity  of  an  odd  target  was  not  determined  by  image  features,  but  by 
amodally  completed  surface  shape  (He  and  Nakayama,  1992).  Studies  on  visual 
texture  segregation  also  indicated  that  the  emergence  of  a  defined  texture  region 
was  dictated  not  by  the  image  shape,  but  surface  shape  (He  and  Nakayama, 

1993) .  Finally,  with  respect  to  apparent  motion,  the  correspondence  between 
tokens  is  not  based  on  image  properties,  but  again  on  surface  properties. 

3.  We  demonstrated  two  entirely  new  and  we  believe,  related,  phenomena 
regarding  the  consequences  of  surface  encoding.  First,  we  demonstrated  that 
apparent  motion  correspondence  is  preferentially  made  between  tokens  which 
reside  on  the  same  surface.  Thus  it  appears  that  motion  is  surface-bound  (He 
and  Nakayama,  1994).  Second,  we  show  that  attention  cannot  be  efficiently 


allocated  to  arbitrary  depths  and  extents  in  space,  but  is  linked  to  perceived 
surfaces.  In  visual  search  experiments,  we  show  that  we  cannot  easily  focus 
attention  across  iso-disparity  or  iso-depth  loci  unless  they  are  part  of  a  well- 
formed  surface.  Yet  we  can  easily  spread  our  attention  selectively  across  well- 
formed  surfaces  which  span  extreme  variations  of  stereoscopic  depth.  In  cueing 
experiments,  we  show  that  selective  attention  is  weakened  if  targets  and 
distractors  either  form  a  single  implicit  plane  receding  in  space  or  simply  rest  on 
such  a  common  plane.  We  conclude  that  attentional  deployment  is  not  "content- 
free",  but  spreads  over  perceived  surfaces  (He  and  Nakayama,  unpublished). 

Of  interest  is  the  fact  that  the  two  phenomena  are  very  likely  to  be  related.  Thus, 
the  obvious  conclusion  is  that  visual  motion  is  bound  to  surfaces  as  a 
consequence  of  the  surface-bound  nature  of  visual  attention. 

4.  We  have  uncovered  a  new  and  unexpected  short-term  memory  effect,  the 
priming  of  pop-out  or  attentional  deployment.  We  examined  a  visual  search  task 
in  which  observers  respond  to  the  high-acuity  aspect  of  a  pop-out  target  (shape 
of  an  odd-colored  diamond,  vernier  offset  of  an  odd  spatial  frequency  patch). 
Repetition  of  the  attention-driving  feature  (color,  spatial  frequency)  in  this  task 
primes  the  pop-out;  repetition  of  the  high  acuity  shape/ response  aspect  does  not. 
Priming  of  pop-out  is  due  to  a  decaying  memory  trace  of  the  attention- focusing 
feature  laid  down  with  each  trial.  The  trace  exerts  a  diminishing  effect  over  the 
following  5-8  trials  (~30  seconds)  and  its  influence  over  this  time  is  cumulative. 
Observers  cannot  willfully  overcome  the  priming,  suggesting  it  is  passive  and 
autonomous.  Both  target  facilitation  and  distractor  inhibition  are  evident,  the 
former  having  a  greater  effect.  The  phenomenon  shows  complete  binocular 
transfer  (Maljkovic  and  Nakayama,  1994).  A  similar  effect  can  also  be  seen  if  we 
consider  repeating  positions.  Here,  repetition  of  the  same  target  position  leads  to 
faster  discriminations,  but  the  appearance  of  the  target  at  positions  occupied  by  a 
previous  distractor  slow  down  the  attentional  deployment.  Additional  studies 
indicate  that  the  position  and  feature  priming  are  independent.  Furthermore, 
and  even  more  surprising,  the  priming  is  linear.  That  is,  we  can  predict  the 
changes  in  reaction  time  of  any  complex  sequence  of  previous  trials  simply  by 
assuming  superposition  of  the  additive  effects  of  position  and  feature  priming. 
(Maljkovic  and  Nakayama,  unpublished  dissertation ). 

The  research  mentioned  above  on  pop-out  culminated  in  Vera  Maljkovich's 
dissertation,  which  has  been  appended. 
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